Engineering MLOps by Emmanuel Raj

Author:Emmanuel Raj
Language: eng
Format: epub
Publisher: Packt Publishing Pvt Ltd
Published: 2021-04-16T00:00:00+00:00

Figure 6.3: Traditional versus serverless deployments

To develop serverless applications, the developer only has to focus on the application's logic and not worry about backend or security code, which is taken care of by the cloud services upon deploying serverless applications.

Model streaming

Model streaming is a method of serving models for handling streaming data. There is no beginning or end of streaming data. Every second, data is produced from thousands of sources and must be processed and analyzed as soon as possible. For example, Google Search results must be processed in real time. Model streaming is another way of deploying ML models. It has two main advantages over other model serving techniques, such as REST APIs or batch processing approaches. The first advantage is asynchronicity (serving multiple requests at a time). REST API ML applications are robust and scalable but have the limitation of being synchronous (they process requests from the client on a first come, first serve basis), which can lead to high latency and resource utilization. To cope with this limitation, stream processing is available. It is inherently asynchronous as the user or client does not have to coordinate or wait for the system to process the request.

Stream processing is able to process asynchronously and serve the users on the go. In order to do so, stream processing uses a message broker to receive messages from the users or clients. The message broker allows the data as it comes and spreads the processing over time. The message broker decouples the incoming requests and facilitates communication between the users or clients and the service without being aware of each other's operations, as shown in figure 5.4. There are a couple of options for message streaming brokers, such as Apace Storm, Apache Kafka, Apache Spark, Apache Flint, Amazon Kinesis, and StreamSQL. The tool you choose is dependent on the IT setup and architecture.

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.